v3.23.1

Compare Source

Performance Improvements

• Decrease the threshold in which we use the BQ Storage Read API (#​1925) (eaa1a52)

v3.23.0

Compare Source

Features

• Adds timer decorator to facilitate debugging (#​1917) (ea750e0)
• Support insertAll for range (#​1909) (74e75e8)

Bug Fixes

• Add pyarrow version check for range support (#​1914) (a86d7b9)
• Edit presubmit for to simplify configuration (#​1915) (b739596)

v3.22.0

Compare Source

Features

• Support RANGE in queries Part 2: Arrow (#​1868) (5251b5d)

v3.21.0

Compare Source

Features

• Add compression option ZSTD. (#​1890) (5ed9cce)
• Adds billing to opentel (#​1889) (38697fb)
• Support RANGE in queries Part 1: JSON (#​1884) (3634405)

Bug Fixes

• Add types to DatasetReference constructor (#​1601) (bf8861c)
• Creates linting-typing.cfg in presubmit (#​1881) (c852c15)
• Remove duplicate key time_partitioning from Table._PROPERTY_TO_A… (#​1898) (82ae908)
• Retry query jobs that fail even with ambiguous jobs.getQueryResults REST errors (#​1903, #​1900) (1367b58)

Performance Improvements

• Avoid unnecessary API call in QueryJob.result() when job is already finished (#​1900) (1367b58)

v3.20.1

Compare Source

Bug Fixes

• Make pyarrow an optional dependency post-3.20.0 yanked release (#​1879) (21714e1)

v3.20.0

Compare Source

Features

• Add fields parameter to set_iam_policy for consistency with update methods (#​1872) (08b1e6f)

Bug Fixes

• Correct type checking (#​1848) (2660dbd)
• Update error logging when converting to pyarrow column fails (#​1836) (0ac6e9b)
• Updates a number of optional dependencies (#​1864) (c2496a1)
• Use an allowlist instead of denylist to determine when query_and_wait uses jobs.query API (#​1869) (e265db6)

v3.19.0

Compare Source

Features

• Support RANGE query parameters (#​1827) (b359a9a)
• Support range sql (#​1807) (86a45c9)

Bug Fixes

• Add google-auth as a direct dependency (713ce2c)
• Augment universe_domain handling (#​1837) (53c2cbf)
• deps: Require google-api-core>=1.34.1, >=2.11.0 (713ce2c)
• Supplementary fix to env-based universe resolution (#​1844) (b818992)
• Supplementary fix to env-based universe resolution (#​1847) (6dff50f)

v3.18.0

Compare Source

Features

• Support nullable boolean and Int64 dtypes in insert_rows_from_dataframe (#​1816) (ab0cf4c)
• Support slot_ms in QueryPlanEntry (#​1831) (d62cabb)

Bug Fixes

• Keyword rendering and docstring improvements (#​1829) (4dfb920)

Documentation

• samples: Updates to urllib3 constraint for Python 3.7 (#​1834) (b099c32)
• Update client_query_w_named_params.py to use query_and_wait API (#​1782) (89dfcb6)

v3.17.2

Compare Source

Bug Fixes

• Change load_table_from_json autodetect logic (#​1804) (6249032)

Documentation

• Update to use API (#​1781) (81563b0)
• Update client_query_destination_table.py sample to use query_and_wait (#​1783) (68ebbe1)
• Update query_external_sheets_permanent_table.py to use query_and_wait API (#​1778) (a7be88a)
• Update sample for query_to_arrow to use query_and_wait API (#​1776) (dbf10de)
• Update the query destination table legacy file to use query_and_wait API (#​1775) (ef89f9e)
• Update to use query_and_wait in client_query_w_positional_params.py (#​1786) (410f71e)
• Update to use query_and_wait in samples/client_query_w_timestamp_params.py (#​1785) (ba36948)
• Update to_geodataframe to use query_and_wait functionality (#​1800) (1298594)

v3.17.1

Compare Source

Bug Fixes

• Add pyarrow.large_strign to the _ARROW_SCALAR_IDS_TO_BQ map (#​1796) (b402a6d)
• Retry 'job exceeded rate limits' for DDL queries (#​1794) (39f33b2)

v3.17.0

Compare Source

Features

• Support universe resolution (#​1774) (0b5c1d5)

Bug Fixes

• query_and_wait now retains unknown query configuration _properties (#​1793) (4ba4342)
• Raise ValueError in query_and_wait with wrong job_config type (4ba4342)

Documentation

• Remove unused query code sample (#​1769) (1f96439)
• Update snippets.py to use query_and_wait (#​1773) (d90602d)
• Update multiple samples to change query to query_and_wait (#​1784) (d1161dd)
• Update the query with no cache sample to use query_and_wait API (#​1770) (955a4cd)
• Updates query to query and wait in samples/desktopapp/user_credentials.py (#​1787) (89f1299)

v3.16.0

Compare Source

Features

• Add table_constraints field to Table model (#​1755) (a167f9a)
• Support jsonExtension in LoadJobConfig (#​1751) (0fd7347)

Bug Fixes

• Add detailed message in job error (#​1762) (08483fb)

v3.15.0

Compare Source

Features

• Support JSON type in insert_rows and as a scalar query parameter (#​1757) (02a7d12)
• Support RANGE in schema (#​1746) (8585747)

Bug Fixes

• Deserializing JSON subfields within structs fails (#​1742) (0d93073)
• Due to upstream change in dataset, updates expected results (#​1761) (132c14b)
• Load_table_from_dataframe for higher scale decimal (#​1703) (b9c8be0)
• Updates types-protobuf version for mypy-samples nox session (#​1764) (c0de695)

Performance Improvements

• DB-API uses more efficient query_and_wait when no job ID is provided (#​1747) (d225a94)

v3.14.1

Compare Source

Bug Fixes

• Add missing handler for deserializing json value (#​1587) (09017a9)

v3.14.0

Compare Source

Features

• Add Client.query_and_wait which directly returns a RowIterator of results (#​1722) (89a647e)
• Add job_id, location, project, and query_id properties on RowIterator (#​1733) (494f275)
• Add job_timeout_ms to job configuration classes (#​1675) (84d64cd)
• Add support dataset.max_time_travel_hours (#​1683) (f22eff2)
• Add support for Dataset.isCaseInsensitive (#​1671) (386fa86)
• Add support for Python 3.12 (#​1736) (3c0976a)
• Removed pkg_resources from all test files and moved importlib into pandas extra (#​1726) (1f4ebb1)
• Support data_governance_type (#​1708) (eff365d)

Bug Fixes

• load_table_from_dataframe now assumes there may be local null values (#​1735) (f05dc69)
• Ensure query job retry has longer deadline than API request deadline (#​1734) (5573579)
• Keep RowIterator.total_rows populated after iteration (#​1748) (8482f47)
• Move grpc, proto-plus and protobuf packages to extras (#​1721) (5ce4d13)

Performance Improvements

• Use the first page a results when query(api_method="QUERY") (#​1723) (6290517)

v3.13.0

Compare Source

Features

• Add Model.transform_columns property (#​1661) (5ceed05)
• Add support for dataset.default_rounding_mode (#​1688) (83bc768)

Bug Fixes

• AccessEntry API representation parsing (#​1682) (a40d7ae)

Documentation

• Remove redundant bigquery_update_table_expiration code sample (#​1673) (2dded33)
• Revised create_partitioned_table sample (#​1447) (40ba859)
• Revised relax column mode sample (#​1467) (b8c9276)

v3.12.0

Compare Source

Features

• Add Dataset.storage_billing_model setter, use client.update_dataset(ds, fields=["storage_billing_model"]) to update (#​1643) (5deba50)
• Search statistics (#​1616) (b930e46)
• Widen retry predicate to include ServiceUnavailable (#​1641) (3e021a4)

Bug Fixes

• Allow storage_billing_model to be explicitly set to None to use project default value (#​1665) (514d3e1)
• Relax timeout expectations (#​1645) (1760e94)
• Use isinstance() per E721, unpin flake8 (#​1659) (54a7769)

Documentation

• Revise update_table_expiration sample (#​1457) (03194e0)

v3.11.4

Compare Source

Bug Fixes

• Updates typing in function definitions (#​1613) (db755ce)

v3.11.3

Compare Source

Bug Fixes

• Type annotations include Optional when None is accepted (#​1554) (6c1ab80)

v3.11.2

Compare Source

Bug Fixes

• Updates tests based on revised hacker_news tables (#​1591) (d73cf49)

v3.11.1

Compare Source

Documentation

• Add/reformat return types for cloud RAD docs (#​1582) (6efdce1)

v3.11.0

Compare Source

Features

• Add remote function options to routines (#​1558) (84ad11d)

Bug Fixes

• Filter None values from OpenTelemetry attributes (#​1567) (9ea2e21)
• Handle case when expirationMs is None (#​1553) (fa6e13d)
• Raise most recent exception when not able to fetch query job after starting the job (#​1362) (09cc1df)

v3.10.0

Compare Source

Features

• Add date, datetime, time, timestamp dtype to to_dataframe (#​1547) (64e913d)

v3.9.0

Compare Source

Features

• Expose query job on dbapi cursor (#​1520) (339eb0e)

Bug Fixes

• Keyerror when the load_table_from_dataframe accesses a unmapped dtype dataframe index (#​1535) (a69348a)

v3.8.0

Compare Source

Features

• Add bool, int, float, string dtype to to_dataframe (#​1529) (5e4465d)
• Add default LoadJobConfig to Client (#​1526) (a2520ca)
• Expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob (#​1521) (8270a10)

Bug Fixes

• Loosen ipywidgets restrictions further to address ipython compatibility issues (#​1531) (50e5026)

v3.7.0

Compare Source

Features

• Add connection_properties and create_session to LoadJobConfig (#​1509) (cd0aaa1)
• Add default_query_job_config property and property setter to BQ client (#​1511) (a23092c)

Documentation

• Remove < 3.11 reference from README (#​1502) (c7417f4)

v3.6.0

Compare Source

Features

• Adding preserveAsciiControlCharacter to CSVOptions (#​1491) (f832e7a)

Bug Fixes

• Annotate optional integer parameters with optional type (#​1487) (a190aaa)
• Loosen ipywidget dependency (#​1504) (20d3276)
• Removes scope to avoid unnecessary duplication (#​1503) (665d7ba)

Dependencies

• Update minimum google-cloud-core to 1.6.0 (a190aaa)

v3.5.0

Compare Source

Features

• Add str method to DatasetReference (#​1477) (f32df1f)
• Add preserveAsciiControlCharacter to LoadJobConfig (#​1484) (bd1da9a)

Documentation

• Adds snippet for creating table with external data config (#​1420) (f0ace2a)
• Revise delete label table code sample, add TODO to clean up sni… (#​1466) (0dab7d2)
• samples: Table variable fix (#​1287) (a71888a)

v3.4.2

Compare Source

Bug Fixes

• Add support for python 3.11 (#​1463) (730a1de)
• Require grpcio >= 1.49.1 for python 3.11 (72b25c5)

Dependencies

• Remove upper bound on packaging dependency (#​1440) (6088129)

Documentation

• Create sample to write schema file from table (#​1439) (093cc68)
• Created samples for load table and create table from schema file (#​1436) (8ad2e5b)
• Revise create table cmek sample (#​1452) (57740e4)
• Revise get table labels code sample, add TODO to clean up snipp… (#​1464) (b5ccbfe)
• Revise label table code samples (#​1451) (14ae1f2)
• Revise sample for nested schema (#​1446) (a097631)

v3.4.1

Compare Source

Documentation

• Add info about streaming quota limits to insert_rows* methods (#​1409) (0f08e9a)

Dependencies

• make pyarrow and BQ Storage optional dependencies (e1aa921)

v3.4.0

Compare Source

Features

• Add reference_file_schema_uri to LoadJobConfig, ExternalConfig (#​1399) (931285f)
• Add default value expression (#​1408) (207aa50)
• Add More Specific Type Annotations for Row Dictionaries (#​1295) (eb49873)

v3.3.6

Compare Source

Features

• Reconfigure tqdm progress bar in %%bigquery magic (#​1355) (506f781)

Bug Fixes

• Corrects test for non-existent attribute (#​1395) (a80f436)
• deps: Allow protobuf 3.19.5 (#​1379) (3e4a074)
• deps: Allow pyarrow < 11 (#​1393) (c898546)
• deps: Require requests>=2.21.0 (#​1388) (e398336)
• Refactor to adapt to changes to shapely dependency (#​1376) (2afd278)

Documentation

• Fix typos (#​1372) (21cc525)

Miscellaneous Chores

• release 3.3.6 (4fce1d9)

v3.3.5

Compare Source

Bug Fixes

• deps: Allow protobuf 3.19.5 (#​1379) (3e4a074)

v3.3.3

Compare Source

Bug Fixes

• Refactors code to account for a tdqm code deprecation (#​1357) (1369a9d)
• Validate opentelemetry span job attributes have values (#​1327) (8287af1)

Documentation

• samples: uses function (create_job) more appropriate to the described sample intent (5aeedaa)

v3.3.2

Compare Source

Bug Fixes

• deps: require proto-plus >= 1.22.0 (1de7a52)
• deps: require protobuf >=3.19, < 5.0.0 (#​1311) (1de7a52)

v3.3.1

Compare Source

Bug Fixes

• deps: allow pyarrow < 10 (#​1304) (13616a9)

v3.3.0

Compare Source

Features

• add destination_expiration_time property to copy job (#​1277) (728b07c)

Bug Fixes

• require python 3.7+ (#​1284) (52d9f14)

Documentation

• samples: add table snapshot sample (#​1274) (e760d1b)
• samples: explicitly add bq to samples reqs, upgrade grpc to fix bug on m1 (#​1290) (9b7e3e4)

v3.2.0

Compare Source

Features

• add support for table clones (#​1235) (176fb2a)

Bug Fixes

• deps: proto-plus >= 1.15.0, <2.0.0dev (ba58d3a)
• deps: require packaging >= 14.3, <22.0.0dev (ba58d3a)
• deps: require protobuf>= 3.12.0, <4.0.0dev (#​1263) (ba58d3a)

Documentation

• fix changelog header to consistent size (#​1268) (d03e2a2)

v3.1.0

Compare Source

Features

• add str method to table (#​1199) (8da4fa9)
• refactor AccessEntry to use _properties pattern (#​1125) (acd5612)
• support using BIGQUERY_EMULATOR_HOST environment variable (#​1222) (39294b4)

Bug Fixes

• deps: allow pyarrow v8 (#​1245) (d258690)
• export bigquery.HivePartitioningOptions (#​1217) (8eb757b)
• Skip geography_as_object conversion for REPEATED fields (#​1220) (4d3d6ec)

Documentation

• updated variable typo in comment in code sample (#​1239) (e420112)

v3.0.1

Compare Source

Bug Fixes

• deps: raise exception when pandas is installed but db-dtypes is not (#​1191) (4333910)
• deps: restore dependency on python-dateutil (#​1187) (212d7ec)

v3.0.0

Compare Source

⚠ BREAKING CHANGES

• BigQuery Storage and pyarrow are required dependencies (#​776)
• use nullable Int64 and boolean dtypes in to_dataframe (#​786)
• destination tables are no-longer removed by create_job (#​891)
• In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#​972)
• automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#​972)
• mark the package as type-checked (#​1058)
• default to DATETIME type when loading timezone-naive datetimes from Pandas (#​1061)
• remove out-of-date BigQuery ML protocol buffers (#​1178)

Features

• add api_method parameter to Client.query to select INSERT or QUERY API (#​967) (76d88fb)
• default to DATETIME type when loading timezone-naive datetimes from Pandas (#​1061) (76d88fb)
• destination tables are no-longer removed by create_job (#​891) (76d88fb)
• In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#​972) (76d88fb)
• mark the package as type-checked (#​1058) (76d88fb)
• use StandardSqlField class for Model.feature_columns and Model.label_columns (#​1117) (76d88fb)

Bug Fixes

• automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#​972) (76d88fb)
• improve type annotations for mypy validation (#​1081) (76d88fb)
• remove out-of-date BigQuery ML protocol buffers (#​1178) (76d88fb)
• use nullable Int64 and boolean dtypes in to_dataframe (#​786) (76d88fb)

Documentation

• Add migration guide from version 2.x to 3.x (#​1027) (76d88fb)

Dependencies

• BigQuery Storage and pyarrow are required dependencies (#​776) (76d88fb)